Closed-loop Visual Grasping and Manipulation

Sensors play an important role in providing feedback to robotic control systems. Vision can be an eeective sensing modality due to its speed, low cost, and ex-ibility. It can also serve as an external sensor that can provide control information for devices that lack internal sensing. Many robot grippers do not have internal sensors. These grippers can be prone to error as they operate under open loop control and usually require a precise model of the environment to be effective. This paper describes 2 visual control primi-tives that can be used to provide position control for a sensorless robot system. The primitives use a simple visual tracking and correspondence scheme to provide real-time feedback control in the presence of imprecise camera calibrations. Experimental results are shown for the positioning task of locating, picking up, and inserting a bolt into a nut under visual control. Results are also presented for the visual control of a bolt tightening task.